Hook-timing diagnostics, WhisperX re-alignment, and merge-doc token fixes - #114

Pulls computeCrossCorrelation/findBestLag/validatePeak/nextPowerOfTwo out
of AudioSyncer into pure functions in scripts/lib/audioCorrelation.js, so
the upcoming hook-timing diagnostic script can reuse this exact math for
audio drift detection instead of creating a third copy of it.
AudioSyncer delegates to the shared module; existing tests pass unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds scripts/diagnostics/verify-hook-timing.ts (npm run diagnose:hooks),
the first layer of a reusable diagnostic for the "hooks cut off too
early/late" class of bugs. Recomputes expected hook sections directly
from transcript.json via the canonical getHookSubClips()/hookClipEnd()
in remotion/lib/hookTiming.ts and reports per-hook source windows, frame
ranges, and durations. No media I/O — fast, always-safe first-line check
that would have caught the render-hook-intro.js drift fixed earlier in
this branch before it ever produced a bad render.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds --rendered/--source flags to verify-hook-timing.ts: extracts each
hook's audio from a rendered output and from the original synced source
video, cross-correlates them via the shared FFT utilities, and flags any
hook whose measured drift exceeds --tolerance-ms. This is the black-box
check that catches Remotion encode-time rounding or dropped frames that
the math-consistency layer alone can't see.

New scripts/lib/extractAudioWindow.js handles ffmpeg extraction + wav
loading. Its wavefile import uses a namespace import (not the default-
import-then-destructure pattern AudioSyncer.js uses) — that pattern
throws under Jest's CJS/ESM interop for this package once wavefile isn't
mocked, which AudioSyncer's own mocked tests never exercise.

tests/integration/verify-hook-timing-audio.test.ts validates the whole
extraction+correlation path against real ffmpeg and real (unmocked)
FFT math using a deterministic seeded noise fixture, confirming ~0ms
lag for a matching window and accurate measurement of a deliberately
introduced 50ms offset.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds --verify-content to verify-hook-timing.ts: transcribes each hook's
already-extracted rendered audio via whisper.cpp (Transcriber.js, base.en
model — small/fast, this only needs a rough word list) and diffs it
against the segment's expected spoken words (bounded by its own
hookFrom/hookTo, not the tail-padded sourceEnd). Flags missing/extra
words, catching wrong-phrase bugs — e.g. resolvePhraseToTimeRange
silently falling back to the whole segment — that pure timing checks
can't see. Off by default (slow, may trigger a model download).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

1. verify-hook-timing.ts compared rendered audio against the *unrounded*
   [sourceStart, sourceEnd) window instead of the frame-rounded
   [trimBefore, trimAfter) window Remotion actually renders — a
   systematic up-to-1-frame bias baked into the check itself. Running
   the full pipeline against a real render (npm run render:hook-intro)
   surfaced this: every one of 35 hooks showed an identical ~50-66ms
   lag before the fix, and a perfectly uniform 50.0ms after — the
   uniformity across unrelated hooks was the tell that this was a
   comparison bug, not per-hook drift.

2. extractAudioWindow.js's wavefile import broke depending on how it's
   loaded: a namespace-then-destructure worked under Jest but under tsx
   executing a real .ts file (the actual production path, as opposed to
   `tsx -e`) the CJS/ESM interop wraps the module as { default: {
   WaveFile } } instead of exposing it directly. Neither shape alone
   covers both runners; now defensively checks both.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Flags any hook whose hookFrom/hookTo doesn't overlap any of its own
segment's spoken token timestamps. Found by manually investigating a
user report that a hook's audio didn't match its caption — the
segment's Whisper/WhisperX word alignment was compressed/wrong (all
four words crammed into a 0.4s span that doesn't match natural speech
pacing), so the doc's explicit HOOK timestamp — set correctly by ear —
didn't overlap the (bad) token positions at all. This check catches
that structurally, for free, with no media I/O, at exactly the row the
content-diff layer silently missed (expectedWords was empty for these
rows, so an empty-vs-empty diff falsely "passed").

This is a "verify by ear" signal, not pass/fail: a correctly hand-fixed
boundary still won't overlap the underlying bad tokens, so it will
keep firing on rows that are already fine. Documented as such in the
row output and --help text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds diagnoseHookEnding() to the content-diff layer: compares the
transcribed rendered clip's actual last word against the hook's
intended final word (hookPhrase, or the segment's own text), and on a
mismatch, auto-triages the root cause:

  - code-bug            a token for the word exists inside hookClipEnd's
                         own search window but wasn't used to extend the
                         clip — a real regression to fix in hookTiming.ts.
  - needs-retiming      a token exists outside the window, but the
                         segment's alignment is otherwise trustworthy —
                         widen hookTo (a suggested value is computed).
  - bad-alignment-data  no trustworthy token exists for the word at all —
                         can't be auto-fixed, needs a human/agent to
                         verify the true ending by listening.

The zero-token-overlap check (previous commit) only caught the extreme
case where a segment's entire token set is disjoint from its hookFrom/
hookTo. A follow-up report that other hooks still end too early, plus a
manual spot-check of 5 hooks it didn't flag (4 of 5 genuinely
truncated), showed the more common failure mode is a hook whose tokens
nominally overlap the window but whose hookTo still lands before the
true spoken ending — this check catches that by verifying against the
actual rendered audio's content instead of just token structure.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The previous commit's diagnoseHookEnding() classified an ending mismatch
as a hookClipEnd() code regression whenever the matching token's t_dtw
fell inside hookClipEnd()'s own search window. That reasoning is wrong
by construction: hookClipEnd() always extends sourceEnd to cover
whichever in-window spoken token has the LARGEST t_end, so any token
found by text-matching is necessarily one of the candidates already
considered — its t_end can never exceed what hookClipEnd() used. An
in-window mismatch can therefore never indicate a real code bug; it can
only mean the matched token's own timestamp is wrong.

Confirmed directly on hook #4 of the ragtech transcript: hookClipEnd()
computed sourceEnd=63.466 (called directly to verify), correctly
covering the "makes" token's t_end=63.446 — yet both the real rendered
clip and the raw source video, transcribed independently, say
"...where every time there's a..." trailing into silence. The token
data for "the/agentic/tool/makes" doesn't correspond to real speech at
that position (a stumbled false start got the clean phrase's timestamp,
not a code regression).

Collapses 'code-bug' and 'needs-retiming' into a single
'needs-verification' diagnosis: reports the matched token's timing as a
starting point to check by ear, explicitly not a confirmed fix. Renamed
suggestedHookTo -> candidateHookTo to make that non-guarantee clear in
the API too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Fixing hookFrom/hookTo doesn't fix hook captions: HookOverlay's
buildCaptions() filters a segment's tokens to those whose t_dtw falls
within [hookFrom, hookClipEnd()) — for hooks whose tokens are
compressed entirely outside that window (the same alignment-quality
issue behind the earlier hookFrom/hookTo fixes), buildCaptions()
returns [] and no caption renders for the whole hook, even though
audio/video play correctly. No override field exists on Segment to
substitute caption timing independent of tokens.

Adds scripts/align/realign-hooks.js: re-runs the same WhisperX forced-
alignment primitive the pipeline already uses for the whole episode
(run_whisperx_align.py), scoped to each hook's own window and text.
Uses a purpose-built merge instead of reusing align-transcript.js's
proportional-remap fallback for unmatched tokens — that assumes every
token in a segment belongs within the new window, which is false
whenever hookPhrase is a subset of the segment's words. Confirmed by
hitting the bug directly: remapping unmatched tokens from their old
(already-moved) position on a second run collapsed every token to the
window's end. Matched tokens now get fresh WhisperX timestamps;
everything else is deterministically excluded instead.

Also exports applyAlignment/spawnPython from align-transcript.js for
reuse, and fixes a real bug found while building this: an unmatched
token's stale t_end was never cleared after its t_dtw got remapped
forward, producing an invalid t_end < t_dtw state.

Verified on the real transcript: the zero-token-overlap check
(previously flagging 6 hooks) now reports none across all 36 hooks.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

whisperx.align() can silently truncate its returned words at an
internal sentence/pause boundary, even when given the full segment
text and a window wide enough to cover it, with no error or any other
signal — the returned segment's own text field is just a prefix of the
input. Confirmed directly: a segment "...evaluate. Is that what it
means?" returned words only through "evaluate.", dropping the second
sentence entirely. This is a general whisperx.align() limitation, not
specific to hook alignment — it would silently degrade the
whole-episode alignment pipeline too, for any raw segment spanning
more than one sentence.

Adds align_with_retry() in run_whisperx_align.py: detects incomplete
word coverage and retries aligning just the remaining text within the
remaining window until fully covered or 4 passes are exhausted,
logging a warning if still incomplete (previously silent either way).

Verified against the exact failing case before (5/15 words, silently)
and after (15/15) the fix, then re-ran the full scoped re-alignment
from the previous commit and confirmed zero coverage warnings across
all 33 hooks. Render-and-inspect confirmed the previously-missing
caption now displays.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Fixes two compounding bugs in mergeAlignedWords(), found by
investigating a detailed 16-item caption-accuracy report from watching
the actual hook preview:

1. Whisper's own tokens are frequently BPE sub-word split (" orchest" +
   "rate", " Comp" + "ounding"), but WhisperX's alignment output is
   whole-word ("orchestrate"). Comparing each raw token individually
   against whole words meant neither half of a split word ever
   matched, silently dropping the whole word from captions — exactly
   why "orchestrate" and "Compounding" were missing. Tokens are now
   grouped into words (leading-space heuristic, matching the rest of
   this codebase) before matching, then a matched word's [start, end]
   is distributed evenly across its constituent sub-tokens.

2. Matched words are now clamped to t_dtw >= sourceStart. Confirmed
   precisely: hookFrom - firstToken.t_dtw was exactly 0.500 across six
   unrelated hooks/words — an internal WhisperX context-padding
   behavior, not speech variance. Unclamped, this either dropped a
   word from HookOverlay's caption filter (no t_end to survive the
   early-start fallback) or let it survive while an adjacent
   no-leading-space continuation didn't, concatenating unrelated words
   with no space ("these buzzwords" -> "thesewords"). Safe
   unconditionally: a matched word is by definition part of the
   aligned phrase, so it belongs inside the hook's own window.

Verified against every hook named in the user's report via direct
token inspection; all now show complete, correctly-positioned words.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Root cause of a recurring regression: every hook timing fix from
scripts/align/realign-hooks.js was silently reverted to the original
raw/compressed t_dtw/t_end the moment ANY doc edit triggered a
merge-doc run, even for segments the edit didn't touch. The "preserve
manual edits" step in edit-transcript.js's main() always rebuilt token
timing from the fresh raw parse, using the matched previous-run token
only to carry forward text corrections and the cut flag — never
t_dtw/t_end. On top of that, the match itself was keyed by t_dtw value
+ occurrence count, which necessarily breaks once t_dtw is the thing
being corrected.

Adds mergeTokenFields(t, p, preserveTiming), matching tokens primarily
by array position (stable across a realign-hooks.js run, which only
changes timing, never token count/order) instead of by t_dtw value.
When positions align, t_dtw/t_end now carry forward from the matched
token. Falls back to the original value-based matching (timing not
preserved) only when token count differs — a real re-transcription,
where positional correspondence can't be assumed.

All 113 existing edit-transcript tests plus 50 integration tests pass
unchanged; added 6 new tests for mergeTokenFields covering timing
preservation, the non-preserving fallback, text correction alongside
timing, cut-flag carry-forward, punctuation non-inheritance, and
clearing rather than fabricating a missing t_end.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

mergeTokenFields (added in a739a78) adds timestampOffset back onto a
preserved token's t_dtw *and* t_end so the later single subtraction
pass lands on the intended value. But that later pass only subtracted
off from t_dtw, never t_end — so t_end drifted by +timestampOffset on
every merge-doc run while t_dtw stayed correct. Confirmed live: every
hook's t_end grew by exactly +0.5s (this project's offset) per run.

Extracted the offset-application block into applyTimestampOffset() so
it's unit-testable, and made it subtract off from t_end symmetrically
with t_dtw. Verified idempotence by running merge-doc three times in a
row on the live transcript and diffing hook token timing between each
run — zero drift after this fix (previously drifted every run).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

applyTextPartsToTokens' LCS-mismatch path (doc has more words than
token groups) correctly synthesizes new tokens for the doc's extra
words, but only ever added tokens — it never removed a token group
that no doc word maps to anymore once its replacement was
synthesized. Found via hook #302 ("...harness and loop and
compound"): whisper.cpp misheard "loop and" as "lupin" (tokenized as
" l"+"up"+"in"); the user corrected the doc text, LCS synthesized new
"loop"/"and" tokens, but the leftover "lupin" tokens stayed in the
array uncut, still spelled "lupin", rendering as spoken caption/audio
content the doc no longer asks for ("loop and compound" showed as
"loop and lupin compound"). Not specific to this hook — reproduces
for any word-count-changing correction where LCS finds zero overlap
between old and new spelling.

Marks every token in every LCS-unmatched group as cut: true. Matched
groups already get corrected text in place; unmatched ones now get
removed from output instead of lingering with their old text.

Added a lupin->loop-and regression test; updated the existing "a
lot's"->"award's" test (same code path) to assert the leftover
tokens are now cut rather than merely absent from segment.text.
Also documents the also-newly-added Commit 12/13 entries in
HOOK_TIMING_DIAGNOSTICS.md (token-timing preservation across
merge-doc runs, and the t_end offset-compounding fix), which were
implemented and committed in earlier commits but not yet written up.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…zard resume fixes

- Diarizer.js: assignSpeakers now prefers per-word overlap against diarization
  turns (_tokenLevelSpeaker) before falling back to whole-segment overlap
  (_segmentOverlapSpeaker). Whisper's pre-alignment segment boundaries are
  coarse and include leading/trailing silence, which biased the old
  whole-segment method at fast speaker handoffs.
- extract-speaker-candidates.py: extracted _sample_timestamps() and added
  exclude/index-start/append support so the thumbnail wizard can fetch
  additional candidates for a speaker without regenerating already-seen ones.
- generate-thumbnail.js: detects a Remotion render that exits 0 and writes a
  file but produced no visual content (composition data never loaded) via an
  all-channels-zero check, instead of silently accepting a blank thumbnail.
  Also clears any stale output file before rendering so the check can't see
  a leftover from a previous failed attempt.
- wizard.js: guards against a stale audio file left in the input dir from a
  prior episode by comparing its mtime against the synced video; fixes the
  "optimize" step's resumeStep/redoStepId condition and syncResults reuse
  when resuming a run; skips the redundant speaker-name review prompt when
  names were just assigned in the same run; and wires the thumbnail
  candidate-selection prompt to extract-speaker-candidates.py's new "more"
  flow.
- Adds export-property-spec-ai-eng-buzzwords.ts: generates the poddedit Rust
  compositor's Tier 1 acceptance-baseline properties.json for the
  "ai-eng-buzzwords" fixture episode (multi-angle, real cut checkpoints) —
  writes to the poddedit repo, not this one. Adapted from the existing
  export-property-spec.ts generator; see its header for full provenance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>